Creating a Multimodal Design Environment Using Speech and Sketching
نویسنده
چکیده
Sketches are a commonly used tool in the early stages of design. Our previous system, ASSIST(Alvarado, 2000), lets users sketch in a natural fashion and recognizes mechanical systems. ASSIST also interfaces with a simulation tool to show users their sketch in action. ASSISTANCE(Oltmans, 2001) builds on ASSIST by allowing the user to use additional sketching and voice input to describe the behavior of the mechanical device after it has been sketched.
منابع مشابه
Symmetric Multimodal Interaction in a Dynamic Dialogue
Two important themes in current work on interfaces are multimodal interaction and the use of dialogue. Human multimodal dialogues are symmetric, i.e., both participants communicate multimodally. We describe a proof of concept system that supports symmetric multimodal communication for speech and sketching in the domain of simple mechanical device design. We discuss three major aspects of the co...
متن کاملA Cognitive Approach to Sketch Understanding
Sketching is an interactive process of communication, using drawings and linguistic information to convey spatial and conceptual material. Our work on a computational model of sketching has the goal of both explaining human sketching and creating software that can be a human-like partner in sketching interactions. This focus has lead us to explore a very different set of tradeoffs from those ty...
متن کاملCurrent Discussions on Digital Sketching in the Early Stages of Architectural Design in Education
In the architectural design, designers are focused on the early stages of the design process or conceptual design. The ultimate goal of this stage is to find a solution for an existing problem, investigate design space, or explore an idea. This stage conventionally begins with sketches and diagrams to explore ideas and solutions; the ambiguity and vagueness of conventional freehand sketching ca...
متن کاملWord learning in a multimodal environment
We are creating human machine interfaces which let people communicate with machines using natural modalities including speech and gesture. A problem with current multimodal interfaces is that users are forced to learn the set of words and gestures which the interface understands. We report on a trainable interface which lets the user teach the system words of their choice through natural multim...
متن کاملUprising in “Uprising”: A Multimodal Analysis of Bob Marley’s Lyrics
This paper investigates how the theme of uprising is conveyed in Bob Marley’s final music album by the name “Uprising”. Through the methodological lenses of multimodality, attention is focused on how the album cover design, lexical items, literary devices, and other aesthetic ways such as the titles of the ten songs of the album and their order of arrangement contribute to the overall theme of ...
متن کامل